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1. Introduction 

It has long been known that the Holder type source conditions originally used to obtain 
algebraic convergence rates for Tikhonov regularization are only sufficient, not necessary. 
Consequently, other concepts have been introduced to completely characterize those rates 
such as the spectral decay condition of [1, Theorem 2.1], distance functions [2, p.3], and 
variational source conditions [3]; pointers to the origins of those concepts can be found 
therein. We propose here a more concise condition through the notion of interpolation 
spaces and establish links to the concepts just listed. Specifically, we argue that the 
intermediate spaces (£o,£i)0,q produced by the fC-method of real interpolation [4, 5] 
with fine index q = oo naturally capture the essential behavior of (iterated) Tikhonov 
regularization, that is: convergence rates, converse results, and saturation, in the noise- 
free and the noisy case. This is not unexpected, given the resemblance of the fC-functional 
(1) and the Tikhonov functional (42), but we systematically quantify this connection by 
careful estimates with particular attention to the limiting cases 0 = 0,1. In a similar vein, 
the relationship between near-minimizers for “L-functionals” and Tikhonov regularization 
was highlighted in [6, Chapter 6]. We prove analogous convergence and converse 
results for the Landweber iteration, and comment on the applicability of the discrepancy 
principle as a stopping rule. 

This note consists of two main parts. In the first part (Section 2) we develop the 
required preliminaries of the fC-method of real interpolation, introduce the different 
intermediate subspaces, and describe them and their interrelations using spectral theory 
in Hilbert spaces. In passing, we relate to the concepts of distance functions and 
variational source conditions. In the second part (Section 3) we elaborate on how the 
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convergence rates of Tikhonov regularization and Landweber iteration are characterized 
in terms of the intermediate subspaces. 

We write A<B to mean that there is a constant C > 0 independent of the parameters 
specified by quantifiers such that A < CB. If, in addition, B <A,we write A ~ B. 


2. Preliminaries 

2.1. Interpolation spaces 

Let X be a Banach space (here and henceforth: over the reals). Let c X be another 
Banach space, continuously embedded inX. We write || • ||o := || • || and || • for the norms 
ofX andXj, respectively. The X-functional is defined as 

Xf(x) := inf (||x-Xi||Q + t^||xi||j)^^ 2 , xeX, t > 0 . ( 1 ) 

xieXi 

For real 0 < 6 < 1 and 1 < q < oo the X-method of real interpolation defines intermediate 
subspaces X-^ c [X,Xi)g ^ c X based on the integrability of t K^[x). Here we are 
only interested in the case q = oo with one of the spaces embedded into the other, 
and therefore refer to standard sources such as [4, 5] for general definitions. For any 
0 < 0 < 1 and any x e X set 

||x||e:i := supt“®Xt(x), ( 2 ) 

t >0 

possibly infinite. We point out that we include the limiting cases 9 = 0 and 0 = 1 in 
this definition. The reason for the unusual notation will become apparent in Section 2.6 
where instead of Xj we will consider a family of subspaces X^, c X parametrized by j. 
Define the spaces 

Xe^i := (X,Xi) 0 ,^ := {x ex : Hxllg^i < oo} (3) 

with the norm || • ||g.i. For 0 < 0 < 1, these are Banach spaces. Moreover, the following 
embeddings Xj c Xg.j c X are continuous. The space Xq.j need not coincide with Xg for 
0 = 0,1, see the remarks on the Gagliardo completion in [5, Chapter 5, §1], but it will be 
the case in the more specific setting of Section 2.6. 


2.2. The constant Ng 

For 0 < 0 < 1, the constant 1 < iVg < ^/2 given by 


52(1-0) 

iVg“2 := 0®(1 - 0)^"® = sup =sup^- (4) 

Ae[0,l] s>0 + 1 

will play a recurrent role. By convention, Nq := := 1, making 9 Ng continuous on 

[0,1]. For example, if a, b > 0 then Young’s inequality with exponents p = l/(l — 0) and 
q = 1/0 gives 


N^a^-%^ = (a/(l - 0))^-‘'(b/0)'' <a+b. (5) 

As another example, for any real A > 0 and t > 0, any real k> 1/2, and any 0 < 0 < 1, 


iV, 


2(l-2fc) 


(- 2 ( 1 - 0 ) 
t2 + 


< 


a + A 


2k 


for a=ti'"^ 


Q N 1-1/(2J:) 


0 


( 6 ) 
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Indeed, after algebraic simplification, the inequality in (6) is equivalent to (a + Ay < 
(1 — 9y~PaP + 6^~PAP, itself a consequence of Holder’s inequality with p = 2fc > 1 as one 
of the exponents. 

2.3. Distance function 

Distance functions were introduced as a means to characterize the regularization error of 
linear regularization operators in [2], previously also in [7, Theorem 2.12]. We briefly 
comment on the relation to the IC-functional. Let 0 < 0 < 1. Fix x e X. Define the 
distance function 


d(r) := inf{||x-Xillo : Xj eXj, llxilli < r}, r > 0. (7) 

This function is nonnegative, nonincreasing, bounded by d(0) = ||x||o, and convex. To 
simplify the notation, we shall write ||xii|i < r, or similar, without mentioning that 
Xj eXj. It is clear the d has compact support if and only if x eXj. 

Inspection of the definitions of the X-functional (1) and the distance function (7) 
reveals that 

X.(x) = inf(d(r)2 + ^ + tr) = (8) 

r>0 r>0 

where d* is the Legendre-Fenchel conjugate of d. Thus the distance function (7) 
characterizes the subspace Xg.^ c X. The first characterization, boundedness of 
|t“®d*(—1)|, is obvious from (2) and (8). The second characterization is the behavior 
of d at infinity, more precisely the identity 

E = N-^\\x\\e,,=D, (9) 

where 


£ := sup inf ||x —XiH,!, ®||xi||® and D:=supd(r)^ ®r®. (10) 

r>0 Ihilli —'■ r>0 

From (9) one infers the more qualitative observation that x e Xg-j if and only if the 
distance function (7) exhibits the asymptotic decay rate d(r) = for r oo. 


Proof of (9). The proof is in the three steps: 

a) E < JV^^||x||g.i. Estimating the inf of E as in (5) for a := ||x — XjHq and 

b:=t^x,\\l, 


inf (■■ ■) <N. 


- 1^-0 

a 


supt ®Xt(x). 

t>0 


Here, the condition ||xi||i < r is redundant if t is large enough, so the infimum becomes 
Kfx), explaining the second inequality. Taking the supremum over r > 0 proves the 
claim. 

b) ||x||g.i < NgD. The definition (10) of D implies d(r) < Using this 

in (8), then computing the infimum yields Kfx) < t^NgD. Now multiply by t“® and take 
SUPt>o. 

c) D <E. If D is finite then d(oo) = 0. Given that d is convex, d is strictly decreasing 

(unless where it vanishes). Thus, for each r > 0, the infimum in d(r) is achieved at 
ilxilli = r. Concerning E, we may suppose that the supinf is assumed at ||xi||i = r, 
adjusting r if necessary. Hence, both D and E equal sup,.>Qinf||^jl^=r Ik ~ ^illo~®lkilli • 
This establishes (9). □ 
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2.4. Interpolation inequality of operators 

Let y be a Banach space. Let S : X ^ 7 be a bounded linear operator with norm Q > 0. 
Assume that Sl;^^ : Xj ^ 7 is also a bounded linear operator, with norm > 0. Then, for 
any 0 < 0 < 1, 


IlSxlly <iVeCoi-®cf||x||0^i VxeXg^i. (11) 

Indeed, let x e Xg-j. Given any Xj e Xj, write x = (x — Xj) + Xj, apply the 
triangle inequality with boundedness of S, and estimate by Cauchy-Schwarz: ||Sx||y < 
Qlk - ^lilo + Ciilxilli < t®(Co + t“2c2)i/2 X t"®(||x - xi||2 + t2||xi||2)h2. Inserting an 
infimum over Xj e X^ then a supremum over t > 0 in the second factor, in view of (2) we 
obtain ||Sx||y < t®(CQ + x ||x||g.i- Minimization over t > 0 gives (11). 


2.5. A lemma for measures 

We will call a nonnegative finite measure pt on Borel subsets of [0, oo) a “Borel measure 
on [0, oo)”. For any such p and any real v > 0 we define |||/t|||v by 

IIIpIII^ := sup r^''n[[0, t)). (12) 

t>0 


The following Lemma records two useful properties of ||| • IH^. 
Lemma 2.1. Let pi.be a Borel measure on [0, oo). Let v > 0. Then 


M([0,A)) + A2t 


X-^^dptW < — 

[A,oo) 


-A^ 


VA > 0 Vy > 0, 


(13) 


and 


A-2'-dM(A)<—VA>0 Vr>0, 

J [0,A) ^ 


(14) 


whenever the right-hand-side is finite. 

Proof. Define the left-continuous function/(A) := p([0,A)) for A > 0. Fix A > 0. Writing 
the integral as a Riemann-Stieltjes integral, and integrating by parts we have 


A-^i'd/rCA) 

J [A,oo) 


A-^’-rf/CA) = -A-2>'/(A) -F 2y 


J [A,oo) 


A-2r-ij(-A)dA. 


[A,oo) 


(15) 


Estimating /(A) < A^’'||||U|||^ under the integral, evaluating, and rearranging leads to (13). 
Similarly, 


A-2''dp(A) < 

”* r 

A-2''d/(A)= A-2''/(A) -F 2r 

J [0,A) ^ 

[O.A] J1 


i/(A)dA. (16) 


[0,A] 


Estimating /(A) < A^*^''"''''^|||/r||p and evaluating the integral yields (14). 


□ 
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Alternative proof. Fix A > 0. Ad (13): Consider := [A, oo)n {A > 0 : A > s}. If 
s > thenAj is empty, otherwise Aj = Therefore 


Pi-^^dptW 

J [A,oo) 


' A 

pCAj )ds 

Jo 


rA 


-a-2v([0,a)) + 


iu([0,s 


Jo 


(17) 


Estimating /t([0, t)) < t^''|||p|||^ under the integral and evaluating yields (13). Ad (14): 
The statement is trivial for r = 0, so suppose r > 0. Consider Bj := [0, A)n{A > 0 : A"^’’ > 
s}. Ifs < A“^'' thenBj = [0,A) and /t(Bj < Otherwise 

, so that p(Bj < s“(''+''^/nilMlllv+r- Using this in A“2''d,2(A) = fx[B,)ds yields 
(14). ' □ 


2.6. Spectral theory in Hilbert spaces 

Suppose that X and Y are real Hilbert spaces. Let T : X ^ 7 be a nonzero bounded 
linear operator. Replacing X byX/kerT if necessary (with the usual quotient norm), we 
assume that 


T is injective. (18) 

Let E denote the projection valued spectral measure of T*T. Then E is compactly 
supported in [0,oo) since T is bounded, and injectivity of T is equivalent to B|o} = 0, 
since range(B|;^}) = ker(r*r — A/). For x eX we define the Borel measure on [0,oo) 
by P;f(A) := ||B, 4 x|p and the associated quantity 


WIv := IIIMxIL =stipt ''||B[o.o^ll’ '^^0- (19) 

t>0 

The subset 


:= {x eX : |||x|||^ < oo} 


( 20 ) 


of X is indeed a Banach space equipped with the norm ||| • |||v. A description of this subspace 
as an interpolation space is subsequently given in Proposition 2.2. The definition of X^ is 
inspired by the work [1]. 

For all real y > 0, we define the Banach space X^, c X as 


Xj, := range((r*r)’') with the norm ||x||^ := ||(r*r) ’'x||, xeX^. 
In terms of the spectral measure E, we can write (note Bjo} = 0) 


lkll^ = 


d/ty(A) and ||x|P = 


[0,oo) 


A-2^d/t^(A) VxeXy. 


( 21 ) 


( 22 ) 


[0,oo) 


For any real 0 < v < y, we define the interpolation space 

Xy.y := (X,Xy)y/y_oo with norm || • H^.^, (23) 

and will denote the corresponding X-functional by Kl. One can check (most easily in the 
case that T is compact) that for any x e X and t > 0, 


|X[(x)|2 = 


inf{(l - ef + e^t^Pi-^'^jdp^W = 

e>0 


t^ + A^t 


[0,oo) 


[0,oo) 


dp^(A). (24) 
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For the limiting cases v = 0 and v = 7 we recover from (22) and (24) that 

Xg.y = Xq and X^.y = Xy with equality of norms. (25) 

The different spaces are related by the following Proposition. 

Proposition 2.2. Let 0 < v < 7 . Let x & X. Then 

Vl-v/7||x||,^, < IlIxIL < iV,/,||x||,,, < ||x||, < lir r|r-’'||x||,. (26) 

Hence, the following embeddings are continuous: 


X^ c 


cX^:y = : 


(27) 


where X^.y = Xy with equivalence of norms possibly not uniform in v < y. 

Proof The inequality (26d) follows from the representation (22), restricting the domain 
of integration to [0, ||r*r||]. The inequality (26c) is obtained by identifying s := 
and 0 := v /7 in (4), and using it in (24), viz. 


\\x\\l = sup 

t>0 " 




[0,oo) 


For the inequality (26b) we use the identity 


2’'= iV^ f-2v/r_ 


v/r t2+s2r 

combined with A < s in the first step of 


for t = s’' 


7 — V 


(28) 


(29) 


- 2 v/r 


(24) 


[0,4 


t^ + 


Taking supf>o on the right, then sup^x, on the left gives (26b). Finally, from (24), followed 
by (13), we have 



I|£[o,4^IP+s^ 


A" 2 ’'dp^(A) 


< 


[s,oo) 


Y — V 


11 ^ 


with the choice s := for the last inequality, and this shows (26a). The last claim is a 
combination of (26a) and (26b). □ 


The choice 7 := 2v in (26) leads to the chain of inequalities 

;; 5 ll|-|L<inf||-||.:,<||-|lv: 2 v<V^II|-|llv, (30) 


and therefore, with equivalence of norms uniformly in v > 0. Equality 

between inf^.^^ || • H^.j, and || • \\y. 2 v does not hold in general. However, if pi,, is a Dirac 
measure supported at some Aq > 0 , and v > 0 , then the infimum of 
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over 7 > V is indeed achieved at 7 = 2 v. 

The constants in (26a) and (26b) are sharp. For example, for being the 

Dirac measure at Aq = 1, 


|||x|L = l and = 1. (32) 

On the other hand, for d/r^(A) = 2vA^''“^dx (that this measure is not compactly 
supported is not essential) we find |||x|||v = 1 , while 

7ZV /Y Y 

\\x\\l.=snpt-^^\Kl[x)\^=— -— = (l + o(l))- as v / 7 , (33) 

t>o sm( 7 rv/ 7 ) 7 -v 

so that the norm equivalence in (26) does deteriorate as v 7 . The relation to the finite 
qualification of the (iterated) Tikhonov regularization is discussed in Section 3.3. 

We provide next another illustration of the fact that is in general strictly larger 
than X^, here for v := 1. We will revisit this example in Section 3.5, Example 3.5. 

Example 2.3. Consider the diagonal operator T := diag((n“^^^)„>i) on the sequence 
spaceX := £ 2 (^ 3 . Then T*T has eigenvalues A„ = n~^, n > 1. Letx^ := Then 

Mxt = li„>i so that 

lllx^'lll^ = sup A^V^t([ 0 , A^,]) = supJV^ V n~^ < 00 , (34) 

N>1 N>1 ^ 

yet, llx' llj = Xn>i finite. Therefore x'^ e Xj \Xi. The sup is assumed at iV = 1, 

so that lllx’^lllj = C(3) ^ (1.096)^. 

In [ 8 , Proposition 11] the variational inequality 

313 >0: |{x,m)|</3|Krr)’'m|r''’'||m||i-’'/’' VmeX, (35) 

was shown for 0 < v < 7 to hold if and only if x e X^. We prove a more precise statement, 
in particular including the limiting cases v = 0 and v = 7 . The first part of the proof (the 
inequality “<”) simplifies and sharpens the corresponding part of [ 8 , Proof of Proposition 
11]. The second part (the inequality “>”) draws from [3]. 

Proposition 2.4. Let x eX and 0 < v < 7 . Then 

sup \\[T*Tyco\\~'’^'^\{x,co)\=N^/^\\x\\^,^. (36) 

||<o||=l 

Proof. For v = 0 the statement is trivial due to ||x||o:j. = ||x||. For v = 7 the statement 
follows from [9, Lemma 8.21] and ||x||y.^ = ||x||j,. In both cases, recall Nq=Ni = 1. For 
the remainder of the proof we assume 0 < v < 7 . 

Let m e X and consider the linear mapping S : x (x, to). Then |Sx| < ||to||||x||, 
and |Sx| < ||(r*T)^to||||x||^ for all x eX^,. By the operator interpolation inequality (11) 
we have |{x, to) | < iVv/^||x||y.j,||(r*r)’'to||’'£’'||to||^“'*'£’'. This implies “<” in (36). 

To verify “>” in (36), it suffices to establish the case 7 = 1, then apply it with [T*Ty 
replacing T*T (also in the definitions of the norms in Section 2.6). Thus we assume 
that (35) holds with 0 < v < 7 = 1. To verify iVv||x||y.i < [3 we check D < iV“^/3 for 
the quantity D from (9). In other words we show the bound for 

the distance function d(-) from (7). To that end we combine [3, Proposition 2.10], [3, 
Theorem 4.1] and [3, Theorem 4.5]: The inequality 


|(x,u))| < ^||(r*r)u)|| 2 -« VtueX with ||<u|| = 1, 


(37) 
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with 


K := and p := ^ ^ for some 0 < ^ < 1 and a > 0, (38) 

implies d^(r) < ip)*(—2r) for all r > 0, where (—(p)* is the Legendre-Fenchel 

conjugate of t —(p(0 := —at’^ defined for t > 0. Straightforward but tedious algebra 
yields the desired estimate d(r)^“''r'' < N~^p. □ 

3. Application to linear inverse problems in Hilbert spaces 

3.1. Linear inverse problem 

Let X and Y be Hilbert spaces. Let T : X ^ 7 be a bounded linear operator, with possibly 
nonclosed range. We write || • || for the norm of X and for that of Y. As in Section 2.6, we 
assume that T is injective. Fix y '' e TX and let denote the solution to 

Tx'' = (39) 

Let Xq e X, called a prior, be given. The task is to find an approximation of x^, given y^ 

(noise-free case) or y® y'^ (noisy case) with 

11/-/II <5. (40) 

We use the notation from Section 2.6, including the spectral measure E, the Borel 
measure pi^, the spaces X^,, X^.^, etc. 

3.2. Spectral cut-ojf regularization 

The spectral cut-off regularization of x^ is defined as Xq + oo)(x'^ — Xg) for a parameter 

a > 0. From the definition (19) of ||| • |||y it is immediate that the error of this regularization 
is 


IIU--E[a.oo))/-^o)ll = ll^[o,a)/''-^o)ll < “''llk'-^olllv Va>0 Vv > 0. (41) 

Since there are no restrictions on the possible convergence rate v > 0 (referred to 
as infinite qualification), and no further constants are involved, we may view the 
performance of this regularization as a reference. 

3.3. Tikhonov regularization 

For a > 0, the regularized solution x„ e X in the noise-free case is defined as the unique 
minimizer of the Tikhonov functional 

:= lly^-7’^ll^ + a|k-^oll^, xeX. (42) 

Replacing y^ by y® defines the regularized solution x^ e X in the noisy case. They are 
equivalently characterized by the first order optimality conditions 


Xct = (T*T + aI) ^(T*y'-Faxo) and x^=(T*T + aI) ^(T*y^ + axg). (43) 
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Writing := for the moment, we have ||re^||^ + a:||e^||^ = ([T*T + 

al)e^,e^) = (T*[y''' — y^),e^) < ||y^ — y^llllre^H < ^5^ + ||re®||^, and cancellation 
of IITe® 11^ on both ends gives the error splitting 

||x^- - x^ll < ||xt - xJI + lle^ll < llx-i- - xJI + (44) 

The parameter a > 0 is determined by a parameter choice strategy 

d : (45) 

The one that minimizes a ||x^ — x^|| whenever y^ and y^' are fixed may be considered 
the optimal strategy. We shall suppose that the parameter choice strategy satisfies 

5"2''sup||x'''-x| 5 ~ supa“’'||x'^-x„|| V5>0, (46) 

where the hidden constants do not depend on 5, the exact solution x^, or the prior Xq, 
but may depend on v > 0. Here, sup( 4 oj means the supremum over all y® e 7 which 
satisfy (40). As an example, the a priori parameter choice strategy d defined by 

V d(5,y'i')||x'''-X5(g_y)|| = ^5 V5 > 0 (47) 

satisfies (46). Specifically, the error splitting (44) quickly yields LHS < 2 RHS in (46) and 
an inspection of [1, Proof of Theorem 2.6] yields y(5/\^)^'''''^LHS > RHS. That proof 
also shows that the optimal strategy, see above, satisfies (46). Of course, of practical 
interest are parameter choice strategies that do not access the exact data y^ or the exact 
solution x^ ; in that regard the notion of quasioptimality by Raus & Hiimarik [10] is 
useful, see comments following Proposition 3.1 below. In any case, (46) formalizes the 
equivalence 

“l|x^-Xa( 5 ,y^..)llV5>0” ^ “||x^-xj| Va>0” (48) 

of the error estimates in the noisy and in the noise-free cases, and in the following we 
assume (46), and focus on its RHS. 

From the abstract theory of interpolation, convergence rates of the Tikhonov 
regularization error ||x’'' — x„|| for x^' e when 0 < v < 1 quickly follow. Indeed, 
for a > 0 consider the linear mapping S^, :X —^X, (x^ — Xq) (x^ — x„). From (43), 

Sa = a[T*T + air\ (49) 

so that IlSall < 1. Under the classical source condition 

x' exQ-l-Xi where Xj = range(r*r), (50) 

it is known (and shown below in (53) for v = 1) that 

l|Sa(x'''-Xo)|| < a||x'''-Xo||i. (51) 

The operator interpolation inequality (11) implies 

l|x'' -x„|| = ||S„(x' -Xo)i| < a''JVvl|x'''-Xo||v:i Vx''exo-l-X^.i Va>0. (52) 

The following Proposition shows that Xq+X^.i czX is precisely the (affine) subspace that 
allows those convergence rates, which is the main observation of this note. 
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Proposition 3.1. Let 0 < v < 1. Then 

< supa“'"||x'''-x„|| < ||x''- XqI^.I. (53) 

a>0 

Equalities hold for v = 0 and v = 1. If v = 1 then sup„>o can be replaced by lirria^o. 

The result is a special case of Proposition 3.2 below, and the proof is therefore 
omitted. Several remarks are in order. 

Combining (26a)-(26b) and (53), for 0 < v < 1, we have that ||x^ — x„|| < Ca'' for 
all a > 0 if and only if x^ e Xq + X^. In essence, this was already shown in [1, Theorem 
2.1]. We emphasize, however, that (26a) and (53) yield the more precise upper bound 

||x^ -x„|| < a''-^|||x'' -XolL Va>0. (54) 

In particular, although the rate of convergence of Tikhonov regularization is at least v 
whenever x ' e Xq + X^, the constant in (54) may deteriorate as v y 1 compared to the 
error (41) of the spectral cut-off regularization. This may be interpreted as a quantitative 
description of the finite qualification of Tikhonov regularization, that is its inability to 
provide convergence larger than v = 1. Of course, by (53), the rate of v = 1 does hold 
if (and only if) x'' e Xq d-Xj, but this condition is more restrictive than x^ e Xq -I- Xj, 
cf Proposition 2.2 and Example 2.3. 

From (53) we see that Tikhonov regularization with a parameter choice rule 
satisfying (46) is order optimal on M^ p := {x e X : ||x||^.i < p} if T has nonclosed 
range. Indeed, (46) and (53) imply the error estimate ||x'^ —x?|| < ^ 2 v/t 2 v+i)pi/(_ 2 v+i) 
whenever x'^ e My.pl on the other hand, [11, Prop 3.15] and comments there show that 
this estimate is optimal because M^ p contains the classical source set {x e X : ||x||v < p'} 
by (26c). Now we can invoke [10, Thm 2.3] to assert that any parameter choice rule 
that is strongly quasioptimal in the sense of [10, Def 2.2] will again give rise to an order 
optimal method on M^ p. 

The final statement of Proposition 3.1 implies the saturation result: If ||x^ — x„|| = 
o(a) as a \ 0 then x^ = Xq. An analogous statement holds in the noisy case for any 
parameter choice strategy satisfying (46). 

An additional consequence of (53) is that, for 0 < v < y, 

^ supa“’'||x' -x„.^|| < ||x^'- XqI^.^ (55) 

where x^.y is given by 

X := argmin{|KT*T)»'/2(x''' - x)||2 -I- a||x''' - XqH^}. (56) 

X€X 

This is simply (53) for the operator {t*TY^^ instead of T. Therefore, letting Xq = 0, the 
mapping x’^ sup„>o a“'*'||x'^ —x„.j,|| defines a norm onX^.^ which is equivalent to ||- ||v.^ 
uniformly in 0 < v < y. 

The variational inequality (35) was used as a starting point in [8, Lemma 2(ii) & 
Lemma 7] for an elementary proof of convergence rates for Tikhonov regularization 
without invoking spectral theory. This is certainly of interest, but leads to the natural 
question whether (35) itself can be established in particular cases without spectral theory; 
the characterization (36) in terms of interpolation spaces is a step in that direction. That 
elementary proof, however, seems to be limited to rates v < 1/2 in (52), as it uses the 
variational inequality (35) with y = 1/2. 
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As an example consider the following integral operator on square integrable 
functions X :=Y := Lj of the unit interval (—1,1), 

T:X^Y, (rx)(t) := J^jX(s)ds-I J^jX(s)dsdT. (57) 

One can check that u := T*Tx solves the boundary value problem —u" = x, iz|±i = 0. 
Hence, X^ from (21) is the Sobolev space OHg. Let now y^(t) := \t\ — 1, so that 
(39) holds with x’^(s) := sign(s). This means, solving (39) with noisy data y® amounts 
to taking the derivative of y '' perturbed by L 2 noise. Set v := 1/4. The interpolation 
space X^-i = (X,Xi)i/ 4 oo is the Besov space [12, Prop 1.25 and 1.30]. In view of 
[12, Def 1.12 of B], to check that x' e X^.-^, it suffices to check that Lj step functions 
are in Bj^^CR), but this is immediate from the intrinsic definition [12, Def 1.1]. On the 
other hand, [13, Ch 1, Thm 11.7] implies that t |x(t)p/(t^ — 1) is integrable for any 
X e Xi /4 (denoted by in [13]), which is clearly not the case for xL In summary, 
x’^ e ^(i/ 4 ):i \^i/ 4 , and Proposition 3.1 implies the convergence rate v = 1/4, familiar 
from [2, Example 5]. 

On a more general note, let M be a connected smooth finite-dimensional Riemannian 
manifold with bounded geometry (complete, injectivity radius bounded away from zero, 
bounded covariant derivatives of the curvature tensor). For instance, the flat space R'^ or 
any compact manifold without boundary has bounded geometry. On M, Sobolev and 
Besov B 2 ;^ spaces can be defined intrinsically through local means [14, §1.11.4], and by 
[14, §1.11.3] one has Thus, for T*T with domain X := and 

range Xj = m > 0, we have X^.j = so that the convergence rate v in (52) 

is characterized in terms of Besov smoothness of x'. This covers a wide class of operators 
T, for instance bijective elliptic pseudo-differential operators H* ^ order —m 

such as (/ — 

3.4. Stationary iterated Tikhonov regularization 

Fixing a > 0, for each integer fc > 1, define x„jt as the minimizer of the Tikhonov 
functional x da(x;x„ (._i), where is given by (42) and x„ o := ^o- Thus, each iterate 
is used as a prior for the next one with the same (stationary) choice of the regularization 
parameter a > 0. Then 


- ^a.it = S^Cx ''- Xq) Vfc>l, (58) 

with Sa from (49). Since commutes with powers of (T*T), we obtain from (51) 
the “shift estimate” ||S„(x'^ — Xa)||^_f < a||x'^ — Xo||;t-t-i-iJ repeating this argument for 
each iteration £ = 1,2, ...,fc, of the fc-fold Tikhonov regularization then ||Sa(x'^ —x„jt)ll < 
a^||x’^ — Xollfc. The operator interpolation inequality readily yields the convergence rate 
a'’ for x^ e Xq+X^.^. and 0 < v < fc. The following extension of Proposition 3.1 is more 
precise. 

Proposition 3.2. Let fc > 1 be an integer, let0<v<kbe real. Then 

“^ollr:fc < supa“'"||x'''-x„_j| < ||x'''- XqII^.^. (59) 

a>0 

Ifv = k then sup^^o replaced by lirrif^^o- 
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Proof. From (58) and (49), 



J [0,oo) 



dPx‘<-XoW- 


(60) 


The first inequality in (59) is therefore a consequence of (6) with 6 = v/k, and the 
representation (24) of the interpolation norm. The second inequality in (59) follows 
by identifying a = t^^^, estimating l/(a + < l/(t^ + and invoking the 

representation (24). Finally, if v = fc, then sup„>o(60) = lim3^o(60) by the Lebesgue 
monotone convergence theorem. □ 


Remarks analogous to those following Proposition 3.1 apply. For instance, the fc-fold 
Tikhonov regularization saturates: If ||x'^ — j.|| = o(a^) as a \ 0 then x'^ = Xq. 


3.5. Landweber iteration 
Fix a > 0 with 

0<cr||r*r|| < 1. (61) 

The Landweber iterates x^ are defined by 

:= +aT*[y' — fc e N, with a given Xq&X, (62) 

and x^ by the same iteration with y^ replaced by y^ in the noisy case. One motivation 
for this iteration is that x^ is a fixed point. By induction one finds 

x'— x^ = {l — aT*T){x'— x^_i) = {l — aT*T)^{x^ — Xq) VfceN, (63) 

and similarly the residual representation 

y^-Tx^ = a-TT*)\y^-Txo) and y^ - Tx^^ = [I - TT*fyy^ - Txq). (64) 

The condition (61) therefore guarantees nondivergence of the iterates. 

For the noisy case, an error splitting analogous to (44) is true [11, Lemma 6.2]: 

l|x'''-xf|| < ||x'''-X;tll+6\/fc Vfc>0, (65) 

thus one often “morally” identifies fc with 1/a. We call a mapping 

fc : (5,y^,...)-^fc(5,y^,...) (66) 

a stopping rule, and in analogy to (45)-(46) we shall suppose that it satisfies 

5“2''sup||x'''-x? 5 ~sup(l + fc/v)’'||x'''-Xfc|| V5>0, (67) 

(40) ’■■■•' k>0 

where the hidden constants do not depend on 5, x^, or Xq, but may depend on v > 0. 
The factor (1 + fc/v) is motivated by (71)-(72). The stopping rule fc may be based on the 
knowledge of some of the iterates X;^, for example it may be the smallest fc > 0 for which 
the discrepancy principle 

Ily^-Txfll <5t (68) 

is satisfied with some fixed threshold t > 1, which in particular is not allowed to depend 
on x^. We will comment on this stopping rule at the end of this section. 

We shall show that convergence rates of the Landweber iteration (62) in the noise- 
free case characterize the spaces . Before formalizing this, we need a lemma. 
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Lemma 3.3. The constant 

j - fs + a\° , 

C 2 := supsup I(s,a):=l - (69) 

a>0 s>0 V y Jo 

is finite with C 2 ^ 1.135. 

Proof. First, /(s,a) is well-defined for all real s > 0 and a > 0. The integral, known 
as the beta function [15, §6.2], evaluates to r(s -I- l)r(a -I- l)/r(s -I- a -I-1). One has 
limj^o^C^)'^) = 1 for ^11 a > 0. Moreover, the function s l(s,a) is increasing for 
0 < a < 1 and decreasing for 1 < a. So we need to consider the pointwise limit /(s, a) as 
s ^ 00 for 0 < a < 1. Stirling’s formula [15, §6.1.38] for r(-) yields 

/(a) := lim/(s,a) = a“°r(a-F 1) Vu > 0, (70) 

S^OO 

which clearly is bounded in 0 < a < 1. We find numerically that 7 is maximized at 
a* ^ 0.3164 with 7(a*) 1.288. The constant C 2 is the square root of the latter. □ 

We can now state the announced rate characterization for the Landweber iteration. 

Proposition 3.4. Let be the Landweber iterates generated by (62) with step size 

a > 0 as in (61). Let v > 0 and r >0. Set 

A[;(x'''-Xo):=sup£:^^'''^’'^|Kr*r)''(x'''-X;,)|| where :=- 7 ——— • (71) 

k>o ak + r + v 

Then, 

Cl - ^olllv < A^Cxl - Xo) < C 2 |||x^' - XolL, (72) 

where C 2 is from (69) and 

Cl := inf {(1 - o-f?;,)'' > {^ae^y/ej*'". (73) 

Proof Fix xl e X. We abbreviate p := and A := A[[(x'^ — Xq). The quantity e has 

been defined such as to satisfy 

= argmax/(A) with /(A) := ^^‘''"'■’'^(1 — crA)^*' Vfc>0. (74) 

x>o 

To enhance readability we will assume for the remainder of the proof that Xq = 0 and 
cr = 1. 

For the first inequality of (72), define p^''[A') := f^?.^''dp(A) on Borel subsets 
A c [0, 00 ). Given 0 < A < ||r*r||, determine the integer fc > 0 by < A < 
Then 


p2'-([o,A])<p2--([o,e,])<(l-s,)-2H X^'-H-Xf’^dpW (75) 

J [ 0 ,e^] 

< (1 — (by definition of A) (76) 

< (1 - a2(’-+’')a2 (77) 

< cf^A^^''+''^A^. (78) 
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Since A > 0 was arbitrary, we obtain the bound 


,2r|| 


r+v < q ^A. 


The first inequality of (72) now follows from: 

r 


2 = supt 2'' 

t>0 


[0,t) 




(79) 


(80) 


We now prove the second inequality in (72). In view of (61), the measure p is 
supported on [0, l/cr], and recall that we have assumed cr = 1. So consider 




|Krmx^-x,)||^ = e- 


- Xf'^dpW 


(81) 


[ 0 . 1 ] 


In the case fc = 0, this is majorized by /r([0,1]) < |||/r|||^. Otherwise, defining /(A) := 
iu( [0, A)) and integrating by parts as in the proof of Lemma 2.1, we estimate 


_ -2(r+r) 


(-(A^''(l - Xf^yiWdX 


[ 0 . 1 ] 




2(r+r) 


2fcA^''(l - A)2''-i/(A)dA. 


[ 0 . 1 ] 


(82) 

(83) 


Estimating /(A) < A^’'|||/r|||^, and employing Lemma 3.3 with s := 2k and a := 2(r + v), 
the second inequality in (72) is obtained. This completes the proof of (72). □ 

Unfortunately, the gap between our “constants” in (72) is not robust in v. For 
example, if r = 0, we have 02 / 0 ^ ~ e'' as v ^ 00 . Qualitatively though, (72) means 
that it is necessary and sufficient for the asymptotic convergence rate v that the data be 
in Xy, which explains the notoriously slow convergence of the Landweber iteration. In 
the limit v ^ 00 , the upper estimate in (72) yields the asymptotics 

\\(^T*Ty-x^.)\\<C 2 e~’^[l + lk^v~'^ + 0(_v~^))\\\x"^-X qIW^ as v ^ 00 , (84) 

for any fixed iteration fc > 0. This bound indicates that, for a fixed iteration fc, only a 
limited amount of smoothness can be exploited (cf. [16, Section 3, Example 3]). 

Prom [2, Theorem 1 and Corollary 1], an estimate similar to (72) with r = 0 can 
be obtained if we assume d(r) ~ for sufficiently large r > 0 for the distance 

function (see Section 2.3). 

We believe it is natural for ||| • |||v to appear in (72) rather than the interpolation norm 
II • llv-j,. The following example illustrates this in the limiting case v = 7 . As in the fc-fold 
Tikhonov regularization, however, it might be possible and meaningful to relate the error 
of the fc-th iterate to the || • 11 ^.;^ norm of the data. 

Example 3.5. Let T and be as in Example 2.3. Recall that ||x^||i = 00 , while 
lllx'jlli 1.096. We will estimate A°(x^) for v := 1. Since ||r|| = 1, we set cr := 1 
to satisfy (61). Then, for any integer fc > 0, 

(1 - A) 2 ''d/r^i = (fc + if 2 n-\l - 
J [0,oo) n>l 


e^^||x^-x;t|p = e^2 


(85) 



Tikhonov and Landweber rates by interpolation spaces 


15 


Splitting the sum at n = k, and exploiting in the finite subsum the fact that t 
t“^(l — peaks at t = 1 + |fc we have 

n>l n<k n>k 


for some C > 0 independent of k, meaning that (85) is bounded uniformly in fc > 0. 
We find numerically that |A°(x'^)P = (85)|;j=o (1.096)^, (85)|j.=i (0.5453)^, 

(85)|{.=2 (0.5475)^, and (85) is decreasing for fc > 2 with lim^^^oo (85) = (1/2)^. 

This limit can also be verified using (70). The infimum in is achieved at fc = 2, so 
Cl = 1/3. Thus (72) reads 1/3 x 1.096 < 1.096 < 1.135 x 1.096. If we use any of the 
iterates {n~^^^{l — as the initial guess Xq, then (72) reads 1/3 x l/^/2 < 1/2 < 

1.135 X 1/V2. 

Our primary motivation for introducing r > 0 in (71) was to estimate the number 
of iterations needed for the stopping rule based on the discrepancy principle (68). We 
briefly comment on this. Fix t > 1. In view of (64) and (61) we have 

|lly’^ - Tx^W - IlyS - Txf III < IKy^- - Tx,) - (y® - Txf )|| < 5 (87) 

for each iteration fc > 0. Hence, if ||y^ —Tx^^H <5(t —l)then (68) is satisfied. Using (72) 
with r = 1/2, this is fulfilled for any fc > 0 such that 5(t — 1) < e^'''^^^C 2 |||x'^ — Xgllly. This 
implies at most fc* ~ 5 - 2 /( 2 v+i) iterations until (68) is met, as in [11, Theorem 6.5] but 
with the slightly weaker assumption that the data is in rather than inX^. To reproduce 
the conclusion of [11, Theorem 6.5], we still need to verify that the discrepancy principle 
(68) implies the error rate ||x’^ —x®|| < 52 r/( 2 v+i) j^gj-e we cannot use the inequality [11, 
(4.66)], 

||xt - x,|| < ||xt - Xo||y(2’'«^||y-^ - rx,||2''/(2v+i), (88) 


as in [11, Proof of Theorem 6.5] for the corresponding error rate with data inX^. Instead, 
we estimate 


lk^-x,||^ = 


(1 — aX)^^dp[?i) 


[0,oo) 


< 


d/i(A) + e 


< e 


2v 


A(1 — crA)^*'d/r(A) 


[eu.oo) 


; + c-i||/-rx,||2<54’'/(2v+i)^ 


(89) 

(90) 

(91) 


and using this in the error splitting (65) yields ||x’'’ — x^|| < 52 v/( 2 v+i)_ 


4. Conclusions 

We have introduced and investigated families of Banach spaces and their interrelations: 
the Hilbert scale X^, the interpolation spaces X^.y, and the spaces X^ = X^. 2 v. We have 
shown that the interpolation spaces X^.y are most adequate for the characterization of 
convergence rates in (iterated) Tikhonov regularization, while X^ are better suited for 
the Landweber iteration. This insight should facilitate the verification of the convergence 
rates. For instance, in many situations, X^.y can be understood as (a subspace of) the 
Besov space oo’ which many characterizations are known. 
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